Context Switching On A Network On Chip

ABSTRACT

A network on chip (NOC) that includes IP blocks, routers, memory communications controllers, and network interface controllers, each IP block adapted to the network by an application messaging interconnect including an inbox and an outbox, one or more of the IP blocks including computer processors supporting a plurality of threads, the NOC also including an inbox and outbox controller configured to set pointers to the inbox and outbox, respectively, that identify valid message data for a current thread; and software running in the current thread that, upon a context switch to a new thread, is configured to: save the pointer values for the current thread, and reset the pointer values to identify valid message data for the new thread, where the inbox and outbox controller are further configured to retain the valid message data for the current thread in the boxes until context switches again to the current thread.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of and claims priorityfrom U.S. patent application Ser. No. 12/118,039, filed on May 9, 2008.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention is data processing, or, more specificallyapparatus and methods for data processing with a network on chip(‘NOC’).

2. Description of Related Art

There are two widely used paradigms of data processing; multipleinstructions, multiple data (‘MIMD’) and single instruction, multipledata (‘SIMD’). In MIMD processing, a computer program is typicallycharacterized as one or more threads of execution operating more or lessindependently, each requiring fast random access to large quantities ofshared memory. MIMD is a data processing paradigm optimized for theparticular classes of programs that fit it, including, for example, wordprocessors, spreadsheets, database managers, many forms oftelecommunications such as browsers, for example, and so on.

SIMD is characterized by a single program running simultaneously inparallel on many processors, each instance of the program operating inthe same way but on separate items of data. SIMD is a data processingparadigm that is optimized for the particular classes of applicationsthat fit it, including, for example, many forms of digital signalprocessing, vector processing, and so on.

There is another class of applications, however, including manyreal-world simulation programs, for example, for which neither pure SIMDnor pure MIMD data processing is optimized. That class of applicationsincludes applications that benefit from parallel processing and alsorequire fast random access to shared memory. For that class of programs,a pure MIMD system will not provide a high degree of parallelism and apure SIMD system will not provide fast random access to main memorystores.

SUMMARY OF THE INVENTION

Methods and apparatus for data processing on a network on chip (‘NOC’)that includes IP blocks, routers, memory communications controllers, andnetwork interface controllers, with each IP block adapted to a routerthrough a memory communications controller and a network interfacecontroller, where each memory communications controller controlscommunication between an IP block and memory, each network interfacecontroller controls inter-IP block communications through routers, witheach IP block also adapted to the network by a low latency, highbandwidth application messaging interconnect including an inbox and anoutbox, one or more IP block including a processor supporting aplurality of threads of execution, where the NOC also includes an inboxcontroller configured to set pointers to the inbox that identify validmessage data for a current thread of execution running on a particularprocessor and an outbox controller configured to set pointers to theoutbox that identify valid message data for the current thread ofexecution running on the particular processor, and software running inthe current thread of execution that, upon a context switch to a newthread of execution, is configured to: save the pointer values for thecurrent thread of execution, and reset the pointer values to identifyvalid message data for the new thread, with the inbox and outboxcontroller also configured to retain the valid message data for thecurrent thread in the inbox and outbox until context switches again tothe current thread.

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescriptions of exemplary embodiments of the invention as illustrated inthe accompanying drawings wherein like reference numbers generallyrepresent like parts of exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 sets forth a block diagram of automated computing machinerycomprising an exemplary computer useful in data processing with a NOCaccording to embodiments of the present invention.

FIG. 2 sets forth a functional block diagram of an example NOC accordingto embodiments of the present invention.

FIG. 3 sets forth a functional block diagram of a further example NOCaccording to embodiments of the present invention.

FIG. 4 sets forth a functional block diagram of a further example NOCaccording to embodiments of the present invention.

FIG. 5 sets forth a flow chart illustrating an exemplary method for dataprocessing with a NOC according to embodiments of the present invention.

FIG. 6 sets forth a flow chart illustrating a further exemplary methodfor data processing with a NOC according to embodiments of the presentinvention.

FIG. 7 sets forth a flow chart illustrating a further exemplary methodfor data processing with a NOC according to embodiments of the presentinvention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary apparatus and methods for data processing with a NOC inaccordance with the present invention are described with reference tothe accompanying drawings, beginning with FIG. 1. FIG. 1 sets forth ablock diagram of automated computing machinery comprising an exemplarycomputer (152) useful in data processing with a NOC according toembodiments of the present invention. The computer (152) of FIG. 1includes at least one computer processor (156) or ‘CPU’ as well asrandom access memory (168) (‘RAM’) which is connected through a highspeed memory bus (166) and bus adapter (158) to processor (156) and toother components of the computer (152).

Stored in RAM (168) is an application program (184), a module ofuser-level computer program instructions for carrying out particulardata processing tasks such as, for example, word processing,spreadsheets, database operations, video gaming, stock marketsimulations, atomic quantum process simulations, or other user-levelapplications. Also stored in RAM (168) is an operating system (154).Operating systems useful data processing with a NOC according toembodiments of the present invention include UNIX™, Linux™, MicrosoftXP™, AIX™, IBM's i5/OS™, and others as will occur to those of skill inthe art. The operating system (154) and the application (184) in theexample of FIG. 1 are shown in RAM (168), but many components of suchsoftware typically are stored in non-volatile memory also, such as, forexample, on a disk drive (170).

The example computer (152) includes two example NOCs according toembodiments of the present invention: a video adapter (209) and acoprocessor (157). The video adapter (209) is an example of an I/Oadapter specially designed for graphic output to a display device (180)such as a display screen or computer monitor. Video adapter (209) isconnected to processor (156) through a high speed video bus (164), busadapter (158), and the front side bus (162), which is also a high speedbus.

The example NOC coprocessor (157) is connected to processor (156)through bus adapter (158), and front side buses (162 and 163), which isalso a high speed bus. The NOC coprocessor of FIG. 1 is optimized toaccelerate particular data processing tasks at the behest of the mainprocessor (156). The NOC video adapter and the NOC coprocessor areoptimized for programs that use parallel processing and also requirefast random access to shared memory. The details of the NOC structureand operation are discussed below with reference to FIGS. 2-6.

The example NOC video adapter (209) and NOC coprocessor (157) of FIG. 1each include a NOC according to embodiments of the present invention,including integrated processor (‘IP’) blocks, routers, memorycommunications controllers, and network interface controllers, each IPblock adapted to a router through a memory communications controller anda network interface controller, each memory communications controllercontrolling communication between an IP block and memory, and eachnetwork interface controller controlling inter-IP block communicationsthrough routers. Each IP block is also adapted to the network by a lowlatency, high bandwidth application messaging interconnect comprising aninbox and an outbox. Each IP block also includes a processor supportinga plurality of threads of execution. Each NOC also includes an inboxcontroller configured to set pointers to the inbox that identify validmessage data for a current thread of execution running on a particularprocessor; an outbox controller configured to set pointers to the outboxthat identify valid message data for the current thread of executionrunning on a particular processor; and software running in the currentthread of execution that, upon a context switch to a new thread ofexecution, is configured to: save the pointer values for the currentthread of execution, and reset the pointer values to identify validmessage data for the new thread, where the inbox and outbox controllerfurther configured to retain the valid message data for the currentthread in the inbox and outbox until context switches again to thecurrent thread.

The computer (152) of FIG. 1 includes disk drive adapter (172) coupledthrough expansion bus (160) and bus adapter (158) to processor (156) andother components of the computer (152). Disk drive adapter (172)connects non-volatile data storage to the computer (152) in the form ofdisk drive (170). Disk drive adapters useful in computers for dataprocessing with a NOC according to embodiments of the present inventioninclude Integrated Drive Electronics (‘IDE’) adapters, Small ComputerSystem Interface (‘SCSI’) adapters, and others as will occur to those ofskill in the art. Non-volatile computer memory also may be implementedfor as an optical disk drive, electrically erasable programmableread-only memory (so-called ‘EEPROM’ or ‘Flash’ memory), RAM drives, andso on, as will occur to those of skill in the art.

The example computer (152) of FIG. 1 includes one or more input/output(‘I/O’) adapters (178). I/O adapters implement user-orientedinput/output through, for example, software drivers and computerhardware for controlling output to display devices such as computerdisplay screens, as well as user input from user input devices (181)such as keyboards and mice.

The exemplary computer (152) of FIG. 1 includes a communications adapter(167) for data communications with other computers (182) and for datacommunications with a data communications network (100). Such datacommunications may be carried out serially through RS-232 connections,through external buses such as a Universal Serial Bus (‘USB’), throughdata communications data communications networks such as IP datacommunications networks, and in other ways as will occur to those ofskill in the art. Communications adapters implement the hardware levelof data communications through which one computer sends datacommunications to another computer, directly or through a datacommunications network. Examples of communications adapters useful fordata processing with a NOC according to embodiments of the presentinvention include modems for wired dial-up communications, Ethernet(IEEE 802.3) adapters for wired data communications networkcommunications, and 802.11 adapters for wireless data communicationsnetwork communications.

For further explanation, FIG. 2 sets forth a functional block diagram ofan example NOC (102) according to embodiments of the present invention.The NOC in the example of FIG. 2 is implemented on a ‘chip’ (100), thatis, on an integrated circuit. The NOC (102) of FIG. 2 includesintegrated processor (‘IP’) blocks (104), routers (110), memorycommunications controllers (106), and network interface controllers(108). Each IP block (104) is adapted to a router (110) through a memorycommunications controller (106) and a network interface controller(108). Each memory communications controller controls communicationsbetween an IP block and memory, and each network interface controller(108) controls inter-IP block communications through routers (110).

In the NOC (102) of FIG. 2, each IP block represents a reusable unit ofsynchronous or asynchronous logic design used as a building block fordata processing within the NOC. The term ‘IP block’ is sometimesexpanded as ‘intellectual property block,’ effectively designating an IPblock as a design that is owned by a party, that is the intellectualproperty of a party, to be licensed to other users or designers ofsemiconductor circuits. In the scope of the present invention, however,there is no requirement that IP blocks be subject to any particularownership, so the term is always expanded in this specification as‘integrated processor block.’ IP blocks, as specified here, are reusableunits of logic, cell, or chip layout design that may or may not be thesubject of intellectual property. IP blocks are logic cores that can beformed as ASIC chip designs or FPGA logic designs.

One way to describe IP blocks by analogy is that IP blocks are for NOCdesign what a library is for computer programming or a discreteintegrated circuit component is for printed circuit board design. InNOCs according to embodiments of the present invention, IP blocks may beimplemented as generic gate netlists, as complete special purpose orgeneral purpose microprocessors, or in other ways as may occur to thoseof skill in the art. A netlist is a Boolean-algebra representation(gates, standard cells) of an IP block's logical-function, analogous toan assembly-code listing for a high-level program application. NOCs alsomay be implemented, for example, in synthesizable form, described in ahardware description language such as Verilog or VHDL. In addition tonetlist and synthesizable implementation, NOCs also may be delivered inlower-level, physical descriptions. Analog IP block elements such asSERDES, PLL, DAC, ADC, and so on, may be distributed in atransistor-layout format such as GDSII. Digital elements of IP blocksare sometimes offered in layout format as well.

In the example of FIG. 2, each IP block includes a low latency, highbandwidth application messaging interconnect (107) that adapts the IPblock to the network for purposes of data communications among IPblocks. As described in more detail below, each such messaginginterconnect includes an inbox and an outbox. In addition, each IP blockalso includes a processor supporting a plurality of threads ofexecution. The NOC (102) also includes an inbox controller configured toset pointers to the inbox that identify valid message data for a currentthread of execution running on a particular processor; an outboxcontroller configured to set pointers to the outbox that identify validmessage data for the current thread of execution running on a particularprocessor; and software running in the current thread of execution that,upon a context switch to a new thread of execution, is configured to:save the pointer values for the current thread of execution, and resetthe pointer values to identify valid message data for the new thread,where the inbox and outbox controller further configured to retain thevalid message data for the current thread in the inbox and outbox untilcontext switches again to the current thread.

Each IP block (104) in the example of FIG. 2 is adapted to a router(110) through a memory communications controller (106). Each memorycommunication controller is an aggregation of synchronous andasynchronous logic circuitry adapted to provide data communicationsbetween an IP block and memory. Examples of such communications betweenIP blocks and memory include memory load instructions and memory storeinstructions. The memory communications controllers (106) are describedin more detail below with reference to FIG. 3.

Each IP block (104) in the example of FIG. 2 is also adapted to a router(110) through a network interface controller (108). Each networkinterface controller (108) controls communications through routers (110)between IP blocks (104). Examples of communications between IP blocksinclude messages carrying data and instructions for processing the dataamong IP blocks in parallel applications and in pipelined applications.The network interface controllers (108) are described in more detailbelow with reference to FIG. 3.

Each IP block (104) in the example of FIG. 2 is adapted to a router(110). The routers (110) and links (120) among the routers implement thenetwork operations of the NOC. The links (120) are packets structuresimplemented on physical, parallel wire buses connecting all the routers.That is, each link is implemented on a wire bus wide enough toaccommodate simultaneously an entire data switching packet, includingall header information and payload data. If a packet structure includes64 bytes, for example, including an eight byte header and 56 bytes ofpayload data, then the wire bus subtending each link is 64 bytes wise,512 wires. In addition, each link is bi-directional, so that if the linkpacket structure includes 64 bytes, the wire bus actually contains 1024wires between each router and each of its neighbors in the network. Amessage can includes more than one packet, but each packet fitsprecisely onto the width of the wire bus. If the connection between therouter and each section of wire bus is referred to as a port, then eachrouter includes five ports, one for each of four directions of datatransmission on the network and a fifth port for adapting the router toa particular IP block through a memory communications controller and anetwork interface controller.

Each memory communications controller (106) in the example of FIG. 2controls communications between an IP block and memory. Memory caninclude off-chip main RAM (112), memory (115) connected directly to anIP block through a memory communications controller (106), on-chipmemory enabled as an IP block (114), and on-chip caches. In the NOC ofFIG. 2, either of the on-chip memories (114, 115), for example, may beimplemented as on-chip cache memory. All these forms of memory can bedisposed in the same address space, physical addresses or virtualaddresses, true even for the memory attached directly to an IP block.Memory-addressed messages therefore can be entirely bidirectional withrespect to IP blocks, because such memory can be addressed directly fromany IP block anywhere on the network. Memory (114) on an IP block can beaddressed from that IP block or from any other IP block in the NOC.Memory (115) attached directly to a memory communication controller canbe addressed by the IP block that is adapted to the network by thatmemory communication controller—and can also be addressed from any otherIP block anywhere in the NOC.

The example NOC includes two memory management units (‘MMUs’) (103,109), illustrating two alternative memory architectures for NOCsaccording to embodiments of the present invention. MMU (103) isimplemented with an IP block, allowing a processor within the IP blockto operate in virtual memory while allowing the entire remainingarchitecture of the NOC to operate in a physical memory address space.The MMU (109) is implemented off-chip, connected to the NOC through adata communications port (116). The port (116) includes the pins andother interconnections required to conduct signals between the NOC andthe MMU, as well as sufficient intelligence to convert message packetsfrom the NOC packet format to the bus format required by the externalMMU (109). The external location of the MMU means that all processors inall IP blocks of the NOC can operate in virtual memory address space,with all conversions to physical addresses of the off-chip memoryhandled by the off-chip MMU (109).

In addition to the two memory architectures illustrated by use of theMMUs (103, 109), data communications port (118) illustrates a thirdmemory architecture useful in NOCs according to embodiments of thepresent invention. Port (118) provides a direct connection between an IPblock (104) of the NOC (102) and off-chip memory (112). With no MMU inthe processing path, this architecture provides utilization of aphysical address space by all the IP blocks of the NOC. In sharing theaddress space bi-directionally, all the IP blocks of the NOC can accessmemory in the address space by memory-addressed messages, includingloads and stores, directed through the IP block connected directly tothe port (118). The port (118) includes the pins and otherinterconnections required to conduct signals between the NOC and theoff-chip memory (112), as well as sufficient intelligence to convertmessage packets from the NOC packet format to the bus format required bythe off-chip memory (112).

In the example of FIG. 2, one of the IP blocks is designated a hostinterface processor (105). A host interface processor (105) provides aninterface between the NOC and a host computer (152) in which the NOC maybe installed and also provides data processing services to the other IPblocks on the NOC, including, for example, receiving and dispatchingamong the IP blocks of the NOC data processing requests from the hostcomputer. A NOC may, for example, implement a video graphics adapter(209) or a coprocessor (157) on a larger computer (152) as describedabove with reference to FIG. 1. In the example of FIG. 2, the hostinterface processor (105) is connected to the larger host computerthrough a data communications port (115). The port (115) includes thepins and other interconnections required to conduct signals between theNOC and the host computer, as well as sufficient intelligence to convertmessage packets from the NOC to the bus format required by the hostcomputer (152). In the example of the NOC coprocessor in the computer ofFIG. 1, such a port would provide data communications format translationbetween the link structure of the NOC coprocessor (157) and the protocolrequired for the front side bus (163) between the NOC coprocessor (157)and the bus adapter (158).

For further explanation, FIG. 3 sets forth a functional block diagram ofa further example NOC according to embodiments of the present invention.The example NOC of FIG. 3 is similar to the example NOC of FIG. 2 inthat the example NOC of FIG. 3 is implemented on a chip (100 on FIG. 2),and the NOC (102) of FIG. 3 includes integrated processor (‘IP’) blocks(104), routers (110), memory communications controllers (106), andnetwork interface controllers (108). Each IP block (104) is adapted to arouter (110) through a memory communications controller (106) and anetwork interface controller (108). Each memory communicationscontroller controls communications between an IP block and memory, andeach network interface controller (108) controls inter-IP blockcommunications through routers (110). In the example of FIG. 3, one set(122) of an IP block (104) adapted to a router (110) through a memorycommunications controller (106) and network interface controller (108)is expanded to aid a more detailed explanation of their structure andoperations. All the IP blocks, memory communications controllers,network interface controllers, and routers in the example of FIG. 3 areconfigured in the same manner as the expanded set (122).

In the example of FIG. 3, each IP block (104) includes a computerprocessor (126) and I/O functionality (124). In this example, computermemory is represented by a segment of random access memory (‘RAM’) (128)in each IP block (104). The memory, as described above with reference tothe example of FIG. 2, can occupy segments of a physical address spacewhose contents on each IP block are addressable and accessible from anyIP block in the NOC. The processors (126), I/O capabilities (124), andmemory (128) on each IP block effectively implement the IP blocks asgenerally programmable microcomputers. As explained above, however, inthe scope of the present invention, IP blocks generally representreusable units of synchronous or asynchronous logic used as buildingblocks for data processing within a NOC. Implementing IP blocks asgenerally programmable microcomputers, therefore, although a commonembodiment useful for purposes of explanation, is not a limitation ofthe present invention.

In the example of FIG. 3, each IP block includes a low latency, highbandwidth application messaging interconnect (107) that adapts the IPblock to the network for purposes of data communications among IPblocks. As described in more detail below, each such messaginginterconnect includes an inbox (460) and an outbox (462). In addition,each IP block (104) also includes a processor (126) supporting aplurality of threads of execution. The NOC (102) also includes an inboxcontroller configured to set pointers to the inbox that identify validmessage data for a current thread of execution running on a particularprocessor; an outbox controller configured to set pointers to the outboxthat identify valid message data for the current thread of executionrunning on a particular processor; and software running in the currentthread of execution that, upon a context switch to a new thread ofexecution, is configured to: save the pointer values for the currentthread of execution, and reset the pointer values to identify validmessage data for the new thread, where the inbox and outbox controllerfurther configured to retain the valid message data for the currentthread in the inbox and outbox until context switches again to thecurrent thread.

In the NOC (102) of FIG. 3, each memory communications controller (106)includes a plurality of memory communications execution engines (140).Each memory communications execution engine (140) is enabled to executememory communications instructions from an IP block (104), includingbidirectional memory communications instruction flow (142, 144, 145)between the network and the IP block (104). The memory communicationsinstructions executed by the memory communications controller mayoriginate, not only from the IP block adapted to a router through aparticular memory communications controller, but also from any IP block(104) anywhere in the NOC (102). That is, any IP block in the NOC cangenerate a memory communications instruction and transmit that memorycommunications instruction through the routers of the NOC to anothermemory communications controller associated with another IP block forexecution of that memory communications instruction. Such memorycommunications instructions can include, for example, translationlookaside buffer control instructions, cache control instructions,barrier instructions, and memory load and store instructions.

Each memory communications execution engine (140) is enabled to executea complete memory communications instruction separately and in parallelwith other memory communications execution engines. The memorycommunications execution engines implement a scalable memory transactionprocessor optimized for concurrent throughput of memory communicationsinstructions. The memory communications controller (106) supportsmultiple memory communications execution engines (140) all of which runconcurrently for simultaneous execution of multiple memorycommunications instructions. A new memory communications instruction isallocated by the memory communications controller (106) to a memorycommunications engine (140) and the memory communications executionengines (140) can accept multiple response events simultaneously. Inthis example, all of the memory communications execution engines (140)are identical. Scaling the number of memory communications instructionsthat can be handled simultaneously by a memory communications controller(106), therefore, is implemented by scaling the number of memorycommunications execution engines (140).

In the NOC (102) of FIG. 3, each network interface controller (108) isenabled to convert communications instructions from command format tonetwork packet format for transmission among the IP blocks (104) throughrouters (110). The communications instructions are formulated in commandformat by the IP block (104) or by the memory communications controller(106) and provided to the network interface controller (108) in commandformat. The command format is a native format that conforms toarchitectural register files of the IP block (104) and the memorycommunications controller (106). The network packet format is the formatrequired for transmission through routers (110) of the network. Eachsuch message is composed of one or more network packets. Examples ofsuch communications instructions that are converted from command formatto packet format in the network interface controller include memory loadinstructions and memory store instructions between IP blocks and memory.Such communications instructions may also include communicationsinstructions that send messages among IP blocks carrying data andinstructions for processing the data among IP blocks in parallelapplications and in pipelined applications.

In the NOC (102) of FIG. 3, each IP block is enabled to sendmemory-address-based communications to and from memory through the IPblock's memory communications controller and then also through itsnetwork interface controller to the network. A memory-address-basedcommunications is a memory access instruction, such as a loadinstruction or a store instruction, that is executed by a memorycommunication execution engine of a memory communications controller ofan IP block. Such memory-address-based communications typicallyoriginate in an IP block, formulated in command format, and handed offto a memory communications controller for execution.

Many memory-address-based communications are executed with messagetraffic, because any memory to be accessed may be located anywhere inthe physical memory address space, on-chip or off-chip, directlyattached to any memory communications controller in the NOC, orultimately accessed through any IP block of the NOC—regardless of whichIP block originated any particular memory-address-based communication.All memory-address-based communication that are executed with messagetraffic are passed from the memory communications controller to anassociated network interface controller for conversion (136) fromcommand format to packet format and transmission through the network ina message. In converting to packet format, the network interfacecontroller also identifies a network address for the packet independence upon the memory address or addresses to be accessed by amemory-address-based communication. Memory address based messages areaddressed with memory addresses. Each memory address is mapped by thenetwork interface controllers to a network address, typically thenetwork location of a memory communications controller responsible forsome range of physical memory addresses. The network location of amemory communication controller (106) is naturally also the networklocation of that memory communication controller's associated router(110), network interface controller (108), and IP block (104). Theinstruction conversion logic (136) within each network interfacecontroller is capable of converting memory addresses to networkaddresses for purposes of transmitting memory-address-basedcommunications through routers of a NOC.

Upon receiving message traffic from routers (110) of the network, eachnetwork interface controller (108) inspects each packet for memoryinstructions. Each packet containing a memory instruction is handed tothe memory communications controller (106) associated with the receivingnetwork interface controller, which executes the memory instructionbefore sending the remaining payload of the packet to the IP block forfurther processing. In this way, memory contents are always prepared tosupport data processing by an IP block before the IP block beginsexecution of instructions from a message that depend upon particularmemory content.

In the NOC (102) of FIG. 3, each IP block (104) is enabled to bypass itsmemory communications controller (106) and send inter-IP block,network-addressed communications (146) directly to the network throughthe IP block's network interface controller (108). Network-addressedcommunications are messages directed by a network address to another IPblock. Such messages transmit working data in pipelined applications,multiple data for single program processing among IP blocks in a SIMDapplication, and so on, as will occur to those of skill in the art. Suchmessages are distinct from memory-address-based communications in thatthey are network addressed from the start, by the originating IP blockwhich knows the network address to which the message is to be directedthrough routers of the NOC. Such network-addressed communications arepassed by the IP block through it I/O functions (124) directly to the IPblock's network interface controller in command format, then convertedto packet format by the network interface controller and transmittedthrough routers of the NOC to another IP block. Such network-addressedcommunications (146) are bi-directional, potentially proceeding to andfrom each IP block of the NOC, depending on their use in any particularapplication. Each network interface controller, however, is enabled toboth send and receive (142) such communications to and from anassociated router, and each network interface controller is enabled toboth send and receive (146) such communications directly to and from anassociated IP block, bypassing an associated memory communicationscontroller (106).

Each network interface controller (108) in the example of FIG. 3 is alsoenabled to implement virtual channels on the network, characterizingnetwork packets by type. Each network interface controller (108)includes virtual channel implementation logic (138) that classifies eachcommunication instruction by type and records the type of instruction ina field of the network packet format before handing off the instructionin packet form to a router (110) for transmission on the NOC. Examplesof communication instruction types include inter-IP blocknetwork-address-based messages, request messages, responses to requestmessages, invalidate messages directed to caches; memory load and storemessages; and responses to memory load messages, and so on.

Each router (110) in the example of FIG. 3 includes routing logic (130),virtual channel control logic (132), and virtual channel buffers (134).The routing logic typically is implemented as a network of synchronousand asynchronous logic that implements a data communications protocolstack for data communication in the network formed by the routers (110),links (120), and bus wires among the routers. The routing logic (130)includes the functionality that readers of skill in the art mightassociate in off-chip networks with routing tables, routing tables in atleast some embodiments being considered too slow and cumbersome for usein a NOC. Routing logic implemented as a network of synchronous andasynchronous logic can be configured to make routing decisions as fastas a single clock cycle. The routing logic in this example routespackets by selecting a port for forwarding each packet received in arouter. Each packet contains a network address to which the packet is tobe routed. Each router in this example includes five ports, four ports(121) connected through bus wires (120-A, 120-B, 120-C, 120-D) to otherrouters and a fifth port (123) connecting each router to its associatedIP block (104) through a network interface controller (108) and a memorycommunications controller (106).

In describing memory-address-based communications above, each memoryaddress was described as mapped by network interface controllers to anetwork address, a network location of a memory communicationscontroller. The network location of a memory communication controller(106) is naturally also the network location of that memorycommunication controller's associated router (110), network interfacecontroller (108), and IP block (104). In inter-IP block, ornetwork-address-based communications, therefore, it is also typical forapplication-level data processing to view network addresses as locationof IP block within the network formed by the routers, links, and buswires of the NOC. FIG. 2 illustrates that one organization of such anetwork is a mesh of rows and columns in which each network address canbe implemented, for example, as either a unique identifier for each setof associated router, IP block, memory communications controller, andnetwork interface controller of the mesh or x,y coordinates of each suchset in the mesh.

In the NOC (102) of FIG. 3, each router (110) implements two or morevirtual communications channels, where each virtual communicationschannel is characterized by a communication type. Communicationinstruction types, and therefore virtual channel types, include thosementioned above: inter-IP block network-address-based messages, requestmessages, responses to request messages, invalidate messages directed tocaches; memory load and store messages; and responses to memory loadmessages, and so on. In support of virtual channels, each router (110)in the example of FIG. 3 also includes virtual channel control logic(132) and virtual channel buffers (134). The virtual channel controllogic (132) examines each received packet for its assignedcommunications type and places each packet in an outgoing virtualchannel buffer for that communications type for transmission through aport to a neighboring router on the NOC.

Each virtual channel buffer (134) has finite storage space. When manypackets are received in a short period of time, a virtual channel buffercan fill up—so that no more packets can be put in the buffer. In otherprotocols, packets arriving on a virtual channel whose buffer is fullwould be dropped. Each virtual channel buffer (134) in this example,however, is enabled with control signals of the bus wires to advisesurrounding routers through the virtual channel control logic to suspendtransmission in a virtual channel, that is, suspend transmission ofpackets of a particular communications type. When one virtual channel isso suspended, all other virtual channels are unaffected—and can continueto operate at full capacity. The control signals are wired all the wayback through each router to each router's associated network interfacecontroller (108). Each network interface controller is configured to,upon receipt of such a signal, refuse to accept, from its associatedmemory communications controller (106) or from its associated IP block(104), communications instructions for the suspended virtual channel. Inthis way, suspension of a virtual channel affects all the hardware thatimplements the virtual channel, all the way back up to the originatingIP blocks.

One effect of suspending packet transmissions in a virtual channel isthat no packets are ever dropped in the architecture of FIG. 3. When arouter encounters a situation in which a packet might be dropped in someunreliable protocol such as, for example, the Internet Protocol, therouters in the example of FIG. 3 suspend by their virtual channelbuffers (134) and their virtual channel control logic (132) alltransmissions of packets in a virtual channel until buffer space isagain available, eliminating any need to drop packets. The NOC of FIG.3, therefore, implements highly reliable network communicationsprotocols with an extremely thin layer of hardware.

For further explanation, FIG. 4 sets forth a functional block diagram ofa further example NOC according to embodiments of the present invention.The example NOC of FIG. 4 is similar to the example NOC of FIG. 2 inthat the example NOC of FIG. 4 is implemented on a chip (100 on FIG. 2),and the NOC (102) of FIG. 4 includes integrated processor (‘IP’) blocks(104), routers (110), memory communications controllers (106), andnetwork interface controllers (108). Each IP block (104) is adapted to arouter (110) through a memory communications controller (106) and anetwork interface controller (108). Each memory communicationscontroller controls communications between an IP block and memory, andeach network interface controller (108) controls inter-IP blockcommunications through routers (110). Each IP block in this example alsoincludes a processor (126) supporting a plurality of threads ofexecution (452-458).

In the example of FIG. 4, each IP block includes a low latency, highbandwidth application messaging interconnect (107) that adapts the IPblock to the network for purposes of data communications among IPblocks. The low latency, high bandwidth application messaginginterconnect (107) is an interconnect in the sense that it is composedof sequential and non-sequential logic that connects an IP block (104)to a network interface controller (108) for purposes of datacommunications. The low latency, high bandwidth application messaginginterconnect (107) is a low latency, high bandwidth interconnect in thatit provides a very fast interconnection between the IP block and thenetwork interface controller—so fast because from the point of view ofthe IP block, for outgoing messages, the process of sending a message tothe network interface controller represents a single immediate write tohigh speed local memory in the outbox array (478), and receiving amessage in the IP block (104) from the network interface controller(108) represents a single read operation from a high speed local memoryin the inbox array (470). As described in more detail below, each suchmessaging interconnect (107) includes an inbox (460) and an outbox(462), each controlled by an associated box controller (464, 472). Inthe example of FIG. 4, one set (122) of an IP block (104) adapted to arouter (110) through a memory communications controller (106) andnetwork interface controller (108) is expanded to aid a more detailedexplanation of the structure and operations of the messaginginterconnect (107). All the IP blocks, memory communicationscontrollers, network interface controllers, and routers in the exampleof FIG. 4 are configured in the same manner as the expanded set (122).

In the example NOC of FIG. 4, each outbox (462) includes an array (478)of memory indexed by an outbox write pointer (474) and an outbox readpointer (476). Each outbox (462) also includes an outbox messagecontroller (472). In the example NOC of FIG. 4, the outbox has anassociated thread of execution (456) that is a module of computerprogram instructions executing on a processor of the IP block. Each suchassociated thread of execution (456) is enabled to write message datainto the array (478) and to provide to the outbox message controller(472) message control information, including message destinationidentification and an indication that message data in the array (478) isready to be sent. The message control information, such as destinationaddress or message identification, and other control information such as‘ready to send,’ may be written to registers in the outbox messagecontroller (472) or such information may be written into the array (478)itself as part of the message data, in a message header, messagemeta-data, or the like.

The outbox message controller (472) is implemented as a network ofsequential and non-sequential logic that is enabled to set the outboxwrite pointer (474). The outbox write pointer (474) may be implemented,for example, as a register in the outbox message controller (472) thatstores the memory address of the location in the array where theassociated thread of execution is authorized to write message data. Theoutbox message controller (472) is also enabled to set the outbox readpointer (476). The outbox read pointer (476) may be implemented, forexample, as a register in the outbox message controller (472) thatstores the memory address of the location in the array where the outboxmessage controller is to read its next message data for transmissionover the network from the outbox.

The outbox message controller (472) is also enabled to send to thenetwork message data written into the array (478) by the thread ofexecution (458) associated with the outbox (462). In the NOC (102) ofFIG. 4, each network interface controller (108) is enabled to convertcommunications instructions from command format to network packet formatfor transmission among the IP blocks (104) through routers (110). Thecommunications instructions are formulated in command format by theassociated thread of execution (458) in the IP block (104) and providedby the outbox message controller (472) to the network interfacecontroller (108) in command format. The command format is a nativeformat that conforms to architectural register files of the IP block(104) and the outbox message controller (472). The network packet formatis the format required for transmission through routers (110) of thenetwork. Each such message is composed of one or more network packets.Such communications instructions may include, for example,communications instructions that send messages among IP blocks carryingdata and instructions for processing the data among IP blocks inparallel applications and in pipelined applications.

In the example NOC of FIG. 4, each inbox (460) includes an array (470)of memory indexed by an inbox write pointer (466) and an inbox readpointer (468). Each inbox (460) also includes an inbox messagecontroller (464). The inbox message controller (464) is implemented as anetwork of sequential and non-sequential logic that is enabled to setthe inbox write pointer (466). The inbox write pointer (466) may beimplemented, for example, as a register in the inbox message controller(464) that stores the memory address of the beginning location in thearray (470) where message data from an outbox of another IP block is tobe written. The inbox message controller (464) is also enabled to setthe inbox read pointer (468). The inbox read pointer (468) may beimplemented, for example, as a register in the inbox message controller(464) that stores the memory address of the beginning location in thearray (470) where an associated thread of execution (456) may read thenext message received from an outbox of some other IP block.

In the example NOC of FIG. 4, the inbox has an associated thread ofexecution (456) that is a module of computer program instructionsexecuting on a processor of the IP block. Each such associated thread ofexecution (456) is enabled to read from the array message data sent fromsome other outbox of another IP block. The thread of execution may benotified that message data sent from another outbox of another IP blockhas been written into the array by the message controller through a flagset in a status register, for example.

The inbox message controller (464) is also enabled to receive from thenetwork message data written to the network from an outbox of another IPblock and provide to a thread of execution (456) associated with theinbox (460) the message data received from the network. The inboxmessage controller of FIG. 4 receives from a network interfacecontroller (108) message data from an outbox of some other IP block andwrites the received message data to the array (470). Upon writing thereceived message data to the array, the inbox message controller (464)is also enabled to notify the thread of execution (456) associated withthe inbox that message data has been received from the network by, forexample, setting a data-ready flag in a status register of the inboxmessage controller (464). The associated thread of execution may, forexample, ‘sleep until flag’ before a message load, or a load opcode canbe configured to check a data-ready flag in the inbox messagecontroller.

In the example NOC of FIG. 4, each box controller (464, 472) isconfigured to set pointers to its associated inbox or outbox thatidentify valid message data for a current thread of execution running ona particular processor. Valid message data is message data in an inboxnot yet read by its recipient and message data in an outbox not yetdelivered to its recipient. A ‘current’ thread of execution is a threadof execution running on a particular processor before a context switch.By contrast, a ‘new’ thread of execution is a thread of execution thatruns on the same particular processor after the context switch. The‘current’ thread of execution, after a context switch in which a newthread of execution runs on the particular processor, may be referred toas the sleeping thread.

A context switch is the computing process of storing and restoring thecontext for a thread on a processor such that multiple threads ofexecution can share a single processor. The context switch is a typicalfeature of a multitasking NOC according to embodiments of the presentinvention. A context switch among two or more threads of execution istypically implemented by storing the context (482) of the thread (456)presently in possession of a processor in a stack (480), replacing thestored context in the architectural registers of the processor with thepreviously-stored context of another thread (including the new thread'sinstruction pointer), and continuing at the point in program executionindicated by the new thread's instruction pointer value. The context fora thread of execution may include the contents of one or more of thearchitectural registers currently used for program execution by thethread. Examples of such architectural registers include, for example,an instruction pointer, status flag registers, one or more stackpointers, memory address indexing registers, one or more general purposeregisters, and so on.

Upon a context switch, context for a thread is typically stored in astack (480). The stack (480) is shown here as a segment of RAM (128) onan IP block, illustrating the fact that the stack is implemented as asegment of main memory, which in this architecture can be, not just onthe same IP block, but anywhere in the NOC's hardware memory addressspace, physically associated with the subject IP block, some other IPblock, this router or another router, on or off the NOC.

Pointers to the inbox and outbox that identify valid message data for acurrent thread of execution running on a particular processor arereferred to in this specification as a base pointer and an offsetpointer for a thread of execution. A base pointer for a thread definesthe beginning of a portion of an inbox or outbox memory array containingvalid message data for the thread. An offset pointer for the threaddefines the extent of the portion of the memory array containing validmessage data for the thread. The portion of the memory array containingvalid message data for the current thread (456) may include message dataretained from one or more previous context switches for another threadof execution. Each thread, whether currently executing or sleeping, mayhave associated base and offset pointers. Such pointers may beimplemented and used during execution of a current thread by the boxcontrollers (464) as read (468, 476) and write (466,474) pointers for aninbox (460) and outbox (462) as described above. That is, a base pointer(558) for an inbox (460), implemented as an inbox read pointer (468),may be, for example, a register in the inbox message controller (464)that stores the memory address of the beginning message data location(550) in an array (470) where an associated thread of execution (456)may read the next message received from an outbox of some other IPblock. An offset pointer (556) for an inbox (460), implemented as aninbox write pointer (466), may be, for example, a register in the inboxmessage controller (464) that stores the memory address of the beginningmessage data location (550) in the array (470) where message data froman outbox of another IP block is to be written. A base pointer (562) foran outbox (462), implemented as an outbox read pointer (476), may be,for example, a register in the outbox message controller (472) thatstores the memory address of the message data location (550) in thearray (478) where the outbox message controller (472) is to read itsnext message data for transmission over the network from the outbox(462). And an offset pointer (560) for an outbox (462), implemented as aoutbox write pointer (474), may be, for example, a register in theoutbox message controller (472) that stores the memory address of themessage data location (550) in the array (478) where the associatedthread (456) of execution is authorized to write message data.

In the example NOC of FIG. 4, the current thread of execution (456)includes software (564) running in the thread. Upon a context switch toa new thread of execution, the software (564) running the current threadis configured to save the pointer values for the current thread ofexecution, that is, values of a base and offset pointer for the currentthread, and reset the pointer values to identify valid message data forthe new thread. The software (564) may save the pointer values for thecurrent thread of execution by retrieving from the box controllers, orarchitectural registers associated with box controllers, the currentvalues of the boxes' read and write pointers and storing the values inassociation with a thread identification of the current thread inmemory, such as RAM (128). The software (564) may reset the pointervalues to identify valid message data for the new thread by setting thepointer values for the new thread to point to the origin of the memoryarray if the new thread has not run previously on the particularprocessor and restoring the pointer values for the new thread frommemory if the new thread was run previously on the particular processorbefore a prior context switch.

The box controllers (464, 474) in the example NOC of FIG. 4 areconfigured to retain valid message data for the current thread (456) inthe inbox and outbox until context switches again to the current thread.That is, during execution of the ‘new’ thread, such as, for example,thread (452) in the example of FIG. 4, the new thread may store messagedata in and read message data from an array (470, 478) without affectingvalid message data of a sleeping thread also stored in the same array(470, 478). In this way, valid message data for a sleeping thread isstored between context switches.

Retaining, by the box controllers (464, 474) in the example NOC of FIG.4, valid message data for the current thread (456) in the inbox andoutbox until context switches again to the current thread may includeassociating the valid message data for the current thread with a threadidentifier (554) of the current thread. The box controller (464, 472)may associate the valid message data for the current thread with athread identifier of the current thread through use of valid bits (552)and thread IDs (554) written to registers in the box controller (464,472) or into the array (470, 478) itself as part of the message data, ina message header, message meta-data, or the like.

Retaining, by the box controllers (464, 474) in the example NOC of FIG.4, valid message data for the current thread (456) in the inbox (460)and outbox (462) until context switches again to the current thread(456) may also include setting a base pointer (558, 562) for the newthread so that, as valid message data is consumed by a recipient afterthe context switch, the new thread does not use storage locations in anarray containing valid message data for the current thread. For anoutbox, a recipient includes another IP block. For an inbox, a recipientincludes the non-sleeping, ‘new’ thread of execution.

Retaining by the box controllers (464, 474) in the example NOC of FIG.4, valid message data for the current thread (456) in the inbox (460)and outbox (462) until context switches again to the current thread mayalso include setting an offset pointer (556, 560) for the new thread sothat, as valid message data is deposited after the context switch, thenew thread does not use storage locations in the memory array containingvalid message data for the current thread. For an outbox, valid messagedata may be deposited by the non-sleeping, ‘new’ thread of execution.For an inbox, valid message data may be deposited by another threadexecuting on another IP block, the intended recipient of the validmessage data being the new thread of execution.

For further explanation, consider the example Table 1 below. Table 1represents an example inbox array (470) that includes associated messagedata locations (550), valid bits (552), and thread identifications(554). The message data locations (550) identify elements of the array(470) in which valid message data is stored. The valid bits (552)indicate whether message data stored in an associated message datalocation has been consumed. The thread IDs (554) are uniqueidentifications for threads of execution on a processor, whethercurrently executing or sleeping until a later context switch. Messagedata locations in some embodiments are the width of a packet, typically64 bytes, although a single message, of course, may include manypackets.

TABLE 1 Inbox Array (470) Message Data Valid Bits Thread ID PointersLocations (550) (552) (554) 0 0 1 1 0 1 BasePtrTh1 (558) 2 1 1 3 1 2 4 11 5 1 1 6 1 2 7 1 2 8 1 2 9 1 1 OffsetPtrTh1 (556) 10 0 1 11 1 2 12 0 113 0 2 . . . . . . . . . . . . . . . . . . . . . . . . . . .Each record in the example of Table 1 indicates whether message data fora particular thread, stored in a particular message data location (550),is valid. In the example of Table 1, two threads, having thread ID 1 andthread ID 2, have valid message data in the inbox array (470).

The example inbox array (470) in Table 1 includes a base pointer (558)and offset pointer (556) for a current thread of execution, a threadcharacterized by a thread ID of ‘1’. A base pointer for an inbox definesthe beginning of a portion of an inbox memory array (470) containingvalid message data for the current thread. The base pointer (558) forthread 1, represented in Table 1 as ‘BasePtrTh1,’ is implemented in thisexample as an inbox read pointer (468) that points to the beginningmessage data location (550) in the array (470) where an associatedthread of execution (456) may read the next message received from anoutbox of some other IP block, that is, valid message data ready forconsumption by a current thread running on a particular processor. Anoffset pointer (556) defines the extent of the portion of the memoryarray (470) containing valid message data for the current thread. Theoffset pointer for thread 1, represented in Table 1 as ‘OffsetPtrTh1’ isimplemented in this example as an inbox write pointer that points to thebeginning message data location (550) in the array (470) where messagedata from an outbox of another IP block is to be written. In the exampleof Table 1, the portion of the memory array containing valid messagedata for the current thread (456), thread 1, includes message dataretained from one or more previous context switches for another threadof execution, thread 2.

Retaining the valid message data for the sleeping thread, thread 2, inthe inbox until context switches again to thread 2, may include setting,by an inbox controller, a base pointer (558) for thread 1 so that, asvalid message data is consumed by thread 1, thread 1 does not usestorage locations in an array containing valid message data for thethread 2. Consider, for example, that thread 1 reads message data storedin location 2 from the inbox array (470) in Table 1. Instead of settingthe value of BasePtrTh1 to point to message data location 3, the inboxcontroller determines that the thread ID associated with message datalocation 3 does not match the thread ID of the currently executingthread, thread 1, and sets the value of BasePtrTh1 to point to messagedata location 4, where valid message data for thread 1 is stored. Alsoupon setting the BasePtrTh1 as such, the inbox controller sets the validbit associated with message data location 2 in the memory array (470) tozero in order to reflect that no valid data exists in message datalocation 2.

Retaining the valid message data for thread 2 in the inbox until contextswitches again to thread 2, may also include setting an offset pointerfor the thread 1 so that, as valid message data is deposited in thearray for thread 1, thread 1 does not use storage locations in thememory array containing valid message data for thread 2. Consider, forexample, that message data for thread 1 is received by the inboxcontroller (464) from an outbox of another IP block and stored inmessage data location 10 of the inbox array. Instead of setting thevalue of OffsetPtrTh1 to point to message data location 11, the inboxcontroller determines that valid data exists in message data location11, determines that the thread ID associated with message data location11 does not match the thread ID of the currently executing thread 1, andsets the value of OffsetPtrTh1 to point to message data location 12,where valid message data for thread 1 may be stored. If valid data didnot exist in message data location 11, the inbox controller may set thevalue of the OffsetPtrTh1 to point to message data location 11 andchange the thread ID associated with message data location 11 to thread1. Upon storing, by the inbox controller, valid message data in amessage data storage location, the inbox controller sets the valid bitassociated with the message data storage location to one.

As mentioned above, software (564 on FIG. 4) running in the currentthread, upon a context switch to a new thread, is configured to resetthe pointer values to identify valid message data for the new thread.The software (564 on FIG. 4) may reset the pointer values to identifyvalid message data for the new thread by restoring the pointer valuesfor the new thread from memory if the new thread was run previously onthe particular processor before a prior context switch. The software(564 on FIG. 4) may restore pointer values for the new thread bytransmitting to the box controllers values of previously saved pointersfor the new thread and a notification of the context switch to the newthread. Turning again to the example of Table 1 above for furtherexplanation, consider that the software (564 on FIG. 4) running inthread 1, the current thread, upon a context switch to a new thread,thread 2, resets the pointer values to identify valid message data forthe new thread, thread 2 by restoring the pointer values for thread 2from memory. In this example, the software (564 on FIG. 4) running inthread 1 may restore the pointer values for thread 2 from memory bytransmitting to the inbox controller the value of the previously savedpointers for thread 2, and a notification of the context switch tothread 2. Upon receipt of such, the inbox controller may set the valuesof the inbox read pointer and inbox write pointer to the previouslysaved values of the base pointer and the offset pointer for thread 2. Inthis example, the previously saved value of the base pointer for thread2 points to message data location 3 and the previously saved value ofthe offset pointer for thread 2 points to the message data location 13.

If the new thread has not run previously on the particular processor,resetting the pointer values to identify valid message data for the newthread may include setting, by the software (564 on FIG. 4) running thecurrent thread, the pointer values for the new thread to point to theorigin of the memory array. Setting, by the software (564) running inthe current thread, the pointer values for the new thread to point tothe origin of the memory array may include transmitting to the boxcontrollers a memory location of the origin of each memory array and anotification of the context switch to the new thread. Turning again forfurther explanation to the example of Table 1 above, consider that thesoftware (564) running in thread 1, the current thread, upon a contextswitch to a new thread, thread 3 (not shown), resets the pointer valuesto identify valid message data for the new thread, thread 3 by setting,by the software (564) running thread 1, the pointer values for thethread 3 to point to the origin of the memory array (470). In thisexample, the software (564) running in thread 1 may set the pointervalues for the thread 3 to point to the origin of the memory array (470)by transmitting to the inbox controller (464) a memory location of theorigin of the memory array (470) and a notification of the contextswitch to thread 3. Upon receipt of such, the inbox controller isconfigured to set pointers to the inbox that identify valid message datafor thread 3. That is, the inbox controller is configured to set thevalues of the base pointer and offset pointer for thread 3 to point tothe next message data location in the memory array (470) havingnon-valid data. In Table 1, the next message data location in the memoryarray (470) having non-valid data is memory location 0.

Table 1 includes an example inbox memory array for ease of explanation.Readers of skill in the art will recognize, however, that retainingvalid message data for a current thread and resetting pointer values toidentify valid message data for a new thread are carried out withrespect to an outbox and an outbox memory array in a similar manner tothat described above with respect to the inbox.

For further explanation, FIG. 5 sets forth a flow chart illustrating anexemplary method for data processing with a NOC according to embodimentsof the present invention. The method of FIG. 5 is implemented on a NOCsimilar to the ones described above in this specification, a NOC (102 onFIG. 3) that is implemented on a chip (100 on FIG. 3) with IP blocks(104 on FIG. 3), routers (110 on FIG. 3), memory communicationscontrollers (106 on FIG. 3), and network interface controllers (108 onFIG. 3). Each IP block (104 on FIG. 3) is adapted to a router (110 onFIG. 3) through a memory communications controller (106 on FIG. 3) and anetwork interface controller (108 on FIG. 3). In the method of FIG. 5,each IP block may be implemented as a reusable unit of synchronous orasynchronous logic design used as a building block for data processingwithin the NOC.

The method of FIG. 5 includes controlling (402) by a memorycommunications controller (106 on FIG. 3) communications between an IPblock and memory. In the method of FIG. 5, the memory communicationscontroller includes a plurality of memory communications executionengines (140 on FIG. 3). Also in the method of FIG. 5, controlling (402)communications between an IP block and memory is carried out byexecuting (404) by each memory communications execution engine acomplete memory communications instruction separately and in parallelwith other memory communications execution engines and executing (406) abidirectional flow of memory communications instructions between thenetwork and the IP block. In the method of FIG. 5, memory communicationsinstructions may include translation lookaside buffer controlinstructions, cache control instructions, barrier instructions, memoryload instructions, and memory store instructions. In the method of FIG.5, memory may include off-chip main RAM, memory connected directly to anIP block through a memory communications controller, on-chip memoryenabled as an IP block, and on-chip caches.

The method of FIG. 5 also includes controlling (408) by a networkinterface controller (108 on FIG. 3) inter-IP block communicationsthrough routers. In the method of FIG. 5, controlling (408) inter-IPblock communications also includes converting (410) by each networkinterface controller communications instructions from command format tonetwork packet format and implementing (412) by each network interfacecontroller virtual channels on the network, including characterizingnetwork packets by type.

The method of FIG. 5 also includes transmitting (414) messages by eachrouter (110 on FIG. 3) through two or more virtual communicationschannels, where each virtual communications channel is characterized bya communication type. Communication instruction types, and thereforevirtual channel types, include, for example: inter-IP blocknetwork-address-based messages, request messages, responses to requestmessages, invalidate messages directed to caches; memory load and storemessages; and responses to memory load messages, and so on. In supportof virtual channels, each router also includes virtual channel controllogic (132 on FIG. 3) and virtual channel buffers (134 on FIG. 3). Thevirtual channel control logic examines each received packet for itsassigned communications type and places each packet in an outgoingvirtual channel buffer for that communications type for transmissionthrough a port to a neighboring router on the NOC.

For further explanation, FIG. 6 sets forth a flow chart illustrating afurther exemplary method for data processing with a NOC according toembodiments of the present invention. The method of FIG. 6 is similar tothe method of FIG. 5 in that the method of FIG. 6 is implemented on aNOC similar to the ones described above in this specification, a NOC(102 on FIG. 3) that is implemented on a chip (100 on FIG. 3) with IPblocks (104 on FIG. 3), routers (110 on FIG. 3), memory communicationscontrollers (106 on FIG. 3), and network interface controllers (108 onFIG. 3). Each IP block (104 on FIG. 3) is adapted to a router (110 onFIG. 3) through a memory communications controller (106 on FIG. 3) and anetwork interface controller (108 on FIG. 3).

In the method of FIG. 6, each IP block (104 on FIG. 3) may beimplemented as a reusable unit of synchronous or asynchronous logicdesign used as a building block for data processing within the NOC, andeach IP block is also adapted to the network by a low latency, highbandwidth application messaging interconnect (107 on FIG. 4) comprisingan inbox (460 on FIG. 4) and an outbox (462 on FIG. 4). In the method ofFIG. 6, each outbox (462 on FIG. 4) includes an outbox messagecontroller (472 on FIG. 4) and an array (478 on FIG. 4) for storingmessage data, with the array indexed by an outbox write pointer (474 onFIG. 4) and an outbox read pointer (476 on FIG. 4). In the method ofFIG. 6, each inbox (460 on FIG. 4) includes an inbox message controller(464 on FIG. 4) and an array (470 on FIG. 4) for storing message data,with the array (470 on FIG. 4) indexed by an inbox write pointer (466 onFIG. 4) and an inbox read pointer (468 on FIG. 4).

The method of FIG. 6, like the method of FIG. 5, the following methodssteps which operate in a similar manner as described above with regardto the method of FIG. 5: controlling (402) by each memory communicationscontroller communications between an IP block and memory, controlling(408) by each network interface controller inter-IP block communicationsthrough routers, and transmitting (414) messages by each router (110 onFIG. 3) through two or more virtual communications channels, where eachvirtual communications channel is characterized by a communication type.

In addition to its similarities to the method of FIG. 5, however, themethod of FIG. 6 also includes setting (502) by the outbox messagecontroller the outbox write pointer. The outbox write pointer (474 onFIG. 4) may be implemented, for example, as a register in the outboxmessage controller (472 on FIG. 4) that stores the memory address of thelocation in the array where the associated thread of execution isauthorized to write message data.

The method of FIG. 6 also includes setting (504) by the outbox messagecontroller the outbox read pointer. The outbox read pointer (476 on FIG.4) may be implemented, for example, as a register in the outbox messagecontroller (472 on FIG. 4) that stores the memory address of thelocation in the array where the outbox message controller is to read itsnext message data for transmission over the network from the outbox.

The method of FIG. 6 also includes providing (506), to the outboxmessage controller by the thread of execution, message controlinformation, including destination identification and an indication thatdata in the array is ready to be sent. The message control information,such as destination address or message identification, and other controlinformation such as ‘ready to send,’ may be written to registers in theoutbox message controller (472 on FIG. 4) or such information may bewritten into the array (478 on FIG. 4) itself as part of the messagedata, in a message header, message meta-data, or the like.

The method of FIG. 6 also includes sending (508), by the outbox messagecontroller to the network, message data written into the array by athread of execution associated with the outbox. In the NOC upon whichthe method of FIG. 6 is implemented, each network interface controller(108 on FIG. 4) is enabled to convert communications instructions fromcommand format to network packet format for transmission among the IPblocks (104 on FIG. 4) through routers (110 on FIG. 4). Thecommunications instructions are formulated in command format by theassociated thread of execution (458 on FIG. 4) in the IP block (104 onFIG. 4) and provided by the outbox message controller (472 on FIG. 4) tothe network interface controller (108 on FIG. 4) in command format. Thecommand format is a native format that conforms to architecturalregister files of the IP block (104 on FIG. 4) and the outbox messagecontroller (472 on FIG. 4). The network packet format is the formatrequired for transmission through routers (110 on FIG. 4) of thenetwork. Each such message is composed of one or more network packets.Such communications instructions may include, for example,communications instructions that send messages among IP blocks carryingdata and instructions for processing the data among IP blocks inparallel applications and in pipelined applications.

The method of FIG. 6 also includes setting (510) by the inbox messagecontroller the inbox write pointer. The inbox write pointer (466 on FIG.4) may be implemented, for example, as a register in the inbox messagecontroller (464 on FIG. 4) that stores the memory address of thebeginning location in the array (470 on FIG. 4) where message data froman outbox of another IP block is to be written.

The method of FIG. 6 also includes setting (512) by the inbox messagecontroller the inbox read pointer. The inbox read pointer (468 on FIG.4) may be implemented, for example, as a register in the inbox messagecontroller (464 on FIG. 4) that stores the memory address of thebeginning location in the array (470 on FIG. 4) where an associatedthread of execution (456 on FIG. 4) may read the next message receivedfrom an outbox of some other IP block.

The method of FIG. 6 also includes receiving (514), by the inbox messagecontroller from the network, message data written to the network fromanother outbox of another IP block, and providing (516), by the inboxmessage controller to a thread of execution associated with the inbox,the message data received from the network. The inbox message controller(464 on FIG. 4) is enabled to receive from the network message datawritten to the network from an outbox of another IP block and provide toa thread of execution (456 on FIG. 4) associated with the inbox (460 onFIG. 4) the message data received from the network. The inbox messagecontroller of FIG. 4 receives from a network interface controller (108on FIG. 4) message data from an outbox of some other IP block and writesthe received message data to the array (470 on FIG. 4).

The method of FIG. 6 also includes notifying (518), by the inbox messagecontroller the thread of execution associated with the inbox, thatmessage data has been received from the network. Upon writing thereceived message data to the array, an inbox message controller (464 onFIG. 4) is also enabled to notify the thread of execution (456 on FIG.4) associated with the inbox that message data has been received fromthe network by, for example, setting a data-ready flag in a statusregister of the inbox message controller (464 on FIG. 4). The associatedthread of execution may, for example, ‘sleep until flag’ before amessage load, or a load opcode can be configured to check a data-readyflag in the inbox message controller.

For further explanation, FIG. 7 sets forth a flow chart illustrating afurther exemplary method for data processing with a NOC according toembodiments of the present invention. The method of FIG. 7 isimplemented on a NOC similar to the ones described above in thisspecification, a NOC (102 on FIG. 3) that is implemented on a chip (100on FIG. 3) with IP blocks (104 on FIG. 3), routers (110 on FIG. 3),memory communications controllers (106 on FIG. 3), and network interfacecontrollers (108 on FIG. 3). Each IP block (104 on FIG. 3) is adapted toa router (110 on FIG. 3) through a memory communications controller (106on FIG. 3) and a network interface controller (108 on FIG. 3). Eachmemory communications controller (106 on FIG. 3) controls communicationbetween an IP block and memory and each network interface controller(108 on FIG. 3) controls inter-IP block communications through routers.On a NOC on which the method of FIG. 7 is implemented, each IP block isfurther adapted to the network by a low latency, high bandwidthapplication messaging interconnect (107 on FIG. 4) comprising an inbox(460 on FIG. 4) and an outbox (462 on FIG. 4), and each IP block alsoincludes a processor (126) supporting a plurality of threads (452-458)of execution. Also on a NOC on which the method of FIG. 7 isimplemented, the NOC includes an inbox controller configured to setpointers to the inbox that identify valid message data for a currentthread of execution running on a particular processor and an outboxcontroller configured to set pointers to the outbox that identify validmessage data for a current thread of execution running on a particularprocessor.

Upon a context switch (901) to a new thread of execution, the method ofFIG. 7 includes saving (902) the pointer values for the current threadof execution; resetting (904) the pointer values to identify validmessage data for the new thread; and retaining (910) the valid messagedata for the current thread in the inbox and outbox until contextswitches again to the current thread.

In the method of FIG. 7, resetting (904) the pointer values to identifyvalid message data for the new thread is carried out by setting (906)the pointer values for the new thread to point to the origin of thememory array if the new thread has not run previously on the particularprocessor and restoring (908) the pointer values for the new thread frommemory if the new thread was run previously on the particular processorbefore a prior context switch.

In the method of FIG. 7, retaining (910) the valid message data for thecurrent thread in the inbox and outbox until context switches again tothe current thread is carried out by associating (912) the valid messagedata for the current thread with a thread identifier of the currentthread; setting (914) a base pointer for the new thread so that, asvalid message data is consumed by a recipient after the context switch,the new thread does not use storage locations in an array containingvalid message data for the current thread; and setting (916) an offsetpointer for the new thread so that, as valid message data is depositedafter the context switch, the new thread does not use storage locationsin the memory array containing valid message data for the currentthread.

Exemplary embodiments of the present invention are described largely inthe context of a fully functional computer system for data processing ona NOC. Readers of skill in the art will recognize, however, that thepresent invention also may be embodied in a computer program productdisposed on computer readable media for use with any suitable dataprocessing system. Such computer readable media may be transmissionmedia or recordable media for machine-readable information, includingmagnetic media, optical media, or other suitable media. Examples ofrecordable media include magnetic disks in hard drives or diskettes,compact disks for optical drives, magnetic tape, and others as willoccur to those of skill in the art. Examples of transmission mediainclude telephone networks for voice communications and digital datacommunications networks such as, for example, Ethernets™ and networksthat communicate with the Internet Protocol and the World Wide Web aswell as wireless transmission media such as, for example, networksimplemented according to the IEEE 802.11 family of specifications.Persons skilled in the art will immediately recognize that any computersystem having suitable programming means will be capable of executingthe steps of the method of the invention as embodied in a programproduct. Persons skilled in the art will recognize immediately that,although some of the exemplary embodiments described in thisspecification are oriented to software installed and executing oncomputer hardware, nevertheless, alternative embodiments implemented asfirmware or as hardware are well within the scope of the presentinvention.

It will be understood from the foregoing description that modificationsand changes may be made in various embodiments of the present inventionwithout departing from its true spirit. The descriptions in thisspecification are for purposes of illustration only and are not to beconstrued in a limiting sense. The scope of the present invention islimited only by the language of the following claims.

1. A network on chip (NOC) comprising: integrated processor (IP) blocks,routers, memory communications controllers, and network interfacecontrollers, each IP block adapted to a router through a memorycommunications controller and a network interface controller, eachmemory communications controller controlling communication between an IPblock and memory, each network interface controller controlling inter-IPblock communications through routers, each IP block further adapted to anetwork by a low latency, high bandwidth application messaginginterconnect comprising an inbox, one or more of the IP blocks furthercomprising computer processors supporting a plurality of threads ofexecution, the NOC further comprising: an inbox controller configured toset pointers to identify a portion of a memory array of the inbox thatcontains valid message data for a current thread of execution running ona particular processor; software running in the current thread ofexecution that, upon a context switch from the current thread ofexecution to a new thread of execution, is configured to: save valuescorresponding to the pointers for the current thread of execution, andreset the values corresponding to the pointers to identify valid messagedata for the new thread, the inbox controller further configured toretain the valid message data for the current thread in the inbox untilcontext switches again to the current thread.
 2. The NOC of claim 1wherein valid message data comprises message data in an inbox not yetread by its recipient.
 3. The NOC of claim 1 wherein the pointersfurther comprise a base pointer and an offset pointer, the base pointerdefining a beginning of a portion of a memory array containing validmessage data for the current thread, the offset pointer defining anextent of the portion of the memory array containing valid message datafor the current thread, and the portion of the memory array containingvalid message data for the current thread optionally also includingmessage data retained from one or more previous context switches toanother thread of execution.
 4. The NOC of claim 1 wherein softwarerunning in the current thread of execution configured to reset thevalues corresponding to the pointers to identify valid message data forthe new thread further comprises software running in the current threadof execution configured to: set pointer values for the new thread topoint to an origin of a memory array if the new thread has not runpreviously on the particular processor; and restore pointer values forthe new thread from memory if the new thread was run previously on theparticular processor before a prior context switch.
 5. The NOC of claim1 wherein the inbox controller further configured to retain the validmessage data for the current thread in the inbox until context switchesagain to the current thread further comprises the inbox controllerfurther configured to: associate the valid message data for the currentthread with a thread identifier of the current thread; set a basepointer for the new thread so that, as valid message data is consumed bya recipient after the context switch, the new thread does not usestorage locations in a memory array containing valid message data forthe current thread; and set an offset pointer for the new thread sothat, as valid message data is deposited after the context switch, thenew thread does not use storage locations in the memory array containingvalid message data for the current thread.
 6. The NOC of claim 1 whereineach IP block comprises a reusable unit of synchronous or asynchronouslogic design used as a building block for data processing within theNOC.
 7. A method of data processing on a network on chip (NOC), the NOCcomprising: integrated processor (IP) blocks, routers, memorycommunications controllers, and network interface controllers, each IPblock adapted to a router through a memory communications controller anda network interface controller, each memory communications controllercontrolling communication between an IP block and memory, each networkinterface controller controlling inter-IP block communications throughrouters, each IP block further adapted to a network by a low latency,high bandwidth application messaging interconnect comprising an inbox,one or more IP block further comprising a processor supporting aplurality of threads of execution, the NOC further comprising an inboxcontroller configured to set pointers to identify a portion of a memoryarray of the inbox that contains valid message data for a current threadof execution running on a particular processor, the method comprising:upon a context switch from the current thread of execution to a newthread of execution: saving, by software running in the current threadof execution, values corresponding to the pointers for the currentthread of execution; resetting, by the software running in the currentthread of execution, the values corresponding to the pointers toidentify valid message data for the new thread; and retaining, by theinbox controller, the valid message data for the current thread in theinbox until context switches again to the current thread.
 8. The methodof claim 7 wherein valid message data comprises message data in an inboxnot yet read by its recipient.
 9. The method of claim 7 wherein thepointers further comprise a base pointer and an offset pointer, the basepointer defining a beginning of a portion of a memory array containingvalid message data for the current thread, the offset pointer definingan extent of the portion of the memory array containing valid messagedata for the current thread, and the portion of the memory arraycontaining valid message data for the current thread optionally alsoincluding message data retained from one or more previous contextswitches to another thread of execution.
 10. The method of claim 7wherein resetting values corresponding to the pointers to identify validmessage data for the new thread further comprises: setting pointervalues for the new thread to point to the origin of the memory array ifthe new thread has not run previously on the particular processor; andrestoring pointer values for the new thread from memory if the newthread was run previously on the particular processor before a priorcontext switch.
 11. The method of claim 7 wherein retaining the validmessage data for the current thread in the inbox until context switchesagain to the current thread further comprises: associating the validmessage data for the current thread with a thread identifier of thecurrent thread; setting a base pointer for the new thread so that, asvalid message data is consumed by a recipient after the context switch,the new thread does not use storage locations in a memory arraycontaining valid message data for the current thread; and setting anoffset pointer for the new thread so that, as valid message data isdeposited after the context switch, the new thread does not use storagelocations in the memory array containing valid message data for thecurrent thread.
 12. The method of claim 7 wherein each IP blockcomprises a reusable unit of synchronous or asynchronous logic designused as a building block for data processing within the NOC.
 13. Acomputer program product for data processing on a network on chip (NOC),the NOC comprising: integrated processor (IP) blocks, routers, memorycommunications controllers, and network interface controllers, each IPblock adapted to a router through a memory communications controller anda network interface controller, each memory communications controllercontrolling communication between an IP block and memory, each networkinterface controller controlling inter-IP block communications throughrouters, each IP block further adapted to a network by a low latency,high bandwidth application messaging interconnect comprising an inbox,one or more IP block further comprising a processor supporting aplurality of threads of execution, the NOC further comprising an inboxcontroller configured to set pointers to identify a portion of a memoryarray of the inbox that contains valid message data for a current threadof execution running on a particular processor, the computer programproduct disposed in a computer readable medium, the computer programproduct comprising computer program instructions capable of: saving, bysoftware running in the current thread of execution, valuescorresponding to the pointers for the current thread of execution;resetting, by the software running in the current thread of execution,values corresponding to the pointers to identify valid message data forthe new thread; and retaining, by the inbox controller, the validmessage data for the current thread in the inbox until context switchesagain to the current thread.
 14. The computer program product of claim13 wherein valid message data comprises message data in an inbox not yetread by its recipient.
 15. The computer program product of claim 13wherein the pointers further comprise a base pointer and an offsetpointer, the base pointer defining a beginning of a portion of a memoryarray containing valid message data for the current thread, the offsetpointer defining an extent of the portion of the memory array containingvalid message data for the current thread, and the portion of the memoryarray containing valid message data for the current thread optionallyalso including message data retained from one or more previous contextswitches to another thread of execution.
 16. The computer programproduct of claim 13 wherein resetting the values corresponding to thepointers to identify valid message data for the new thread furthercomprises: setting pointer values for the new thread to point to theorigin of the memory array if the new thread has not run previously onthe particular processor; and restoring pointer values for the newthread from memory if the new thread was run previously on theparticular processor before a prior context switch.
 17. The computerprogram product of claim 13 wherein retaining the valid message data forthe current thread in the inbox until context switches again to thecurrent thread further comprises: associating the valid message data forthe current thread with a thread identifier of the current thread;setting a base pointer for the new thread so that, as valid message datais consumed by a recipient after the context switch, the new thread doesnot use storage locations in a memory array containing valid messagedata for the current thread; and setting an offset pointer for the newthread so that, as valid message data is deposited after the contextswitch, the new thread does not use storage locations in the memoryarray containing valid message data for the current thread.
 18. Thecomputer program product of claim 13 wherein each IP block comprises areusable unit of synchronous or asynchronous logic design used as abuilding block for data processing within the NOC.
 19. The computerprogram product of claim 13 wherein the computer readable mediumcomprises a recordable medium.
 20. The computer program product of claim13 wherein the computer readable medium comprises a transmission medium.